-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[1pt] PR: Clip ocean mask from DEM and extend outlet streams #1121
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Matt helped show me the before and after. Looks good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
During full scale testing, some errors occurred. It does appear that they only fail on HUC2 of 18 and higher but a full analysis was not done.
Here is an example of one fail (and each failing HUC has a different amount of the "Failed to auto identify EPSG: 7"
Full Error:
Get Vector Layers and Subset 19020103
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Failed to auto identify EPSG: 7
Traceback (most recent call last):
File "//./foss_fim/data/wbd/generate_pre_clip_fim_huc8.py", line 329, in huc_level_clip_vectors_to_wbd
subset_vector_layers(
File "/foss_fim/data/wbd/clip_vectors_to_wbd.py", line 240, in subset_vector_layers
nwm_streams = extend_outlet_streams(nwm_streams, wbd_buffer, wbd)
File "/foss_fim/data/wbd/clip_vectors_to_wbd.py", line 64, in extend_outlet_streams
coords = [(coords) for coords in list(row['geometry'].coords)]
File "/usr/local/lib/python3.10/dist-packages/shapely/geometry/base.py", line 910, in coords
raise NotImplementedError(
NotImplementedError: Sub-geometries may have coordinate sequences, but multi-part geometries do not
Each HUC that had at least one EPSG: 7 failed with the sub-geometries error.
This is not a comprehensive list but some of the ones that failed are:18070305
18070201
19020401
18090101
18100203
and lots of others.
Doesn't appear there are any below 18's but not sure, might have been insufficent logging which was upgraded for the 18 and higher list.
Strange, I just ran a couple of the problem HUCs and didn't have any errors:
|
Some NWM streams, particularly in coastal areas, fail to reach the edge of the DEM resulting in reverse flow (#916 (comment)). This issue was resolved by clipping the ocean mask from the buffered WBD and DEM, and any remaining streams that didn't have outlets reaching the edge of the buffered WBD boundary were extended by snapping the end to the nearest point on the buffered WBD.
Note: this does not solve the issue noted at NOAA-OWP/hydrofabric#38
Changes
data/wbd/clip_vectors_to_wbd.py
: Clipslandsea
ocean mask from the buffered WBD and adds a function to extend outlet streams to the buffered WBDdata/wbd/clip_vectors_to_wbd.py
: Updated multi-processing and added more logging.Testing
DEM before (left) and after (right) clipping the ocean mask from the buffered WBD
Flow direction of DEM-derived reaches (red dashed lines) indicated by bold red arrows (before (left) and after (right))
Issuer Checklist (For developer use)
You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.
[_pt] PR: <description>
dev
branch (the default branch), you have a descriptive Feature Branch name using the format:dev-<description-of-change>
(e.g.dev-revise-levee-masking
)dev
branchpre-commit
hooks were run locally/foss_fim/
, run:pytest unit_tests/
)4.x.x.x
Merge Checklist (For Technical Lead use only)